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Abstract 



We show that if M is a compact oriented surface of genus and G is a 
W subgroup of Sym.p^{M) which has an infinite normal solvable subgroup, then 

G is virtually abelian. In particular the centralizer of an infinite order / G 
Symp'^{M) is virtually abelian. Another immediate corollary is that if G is a 
^ solvable subgroup of SympJ;|^(M) then G is virtually abelian. We also prove a 

special case of the Tits Alternative for subgroups of Symp^(5^). 

(N 
> 

^ 1 Introduction 

CO If M is a compact connected oriented surface we let Diffo''(M) denote the C" diffeo- 

morphisms isotopic to the identity. In particular if r = u this denotes the subgroup 

^ of those which are real analytic. Let Symp^(M) denote the subgroup of Diffo'^(M) 

, — I which preserve a smooth volume form /x. We will denote by Cent' (/) and Cent^(/) 

^ the subgroups of Diffo'^(M) and Symp^M), respectively, whose elements commute 

k3l with /. If G is a subgroup of Symp|[^(M) then Cent^(/, G) will denote the the subgroup 

^ of G whose elements commute with /. In this article we wish to address the algebraic 

structure of subgroups of Symp^(M) and our results are largely limited to the case 
of analytic diffeomorphisms when M has genus 0. Our approach is to understand the 
possible dynamic behavior of such diffeomorphisms and use the structure exhibited 
to conclude information about subgroups. 

Definition 1.1. If is a connected manifold, an element / G DifF^(iV) will be said 
to have full support provided A'"\Fix(/) is dense in A^. We will say that / has support 
of finite type if both Fix(/) and \ Fix(/) have finitely many components. We will 
say that a subgroup G of DifT''(A^) has full support of finite type if every non-trivial 
element of G has full support of finite type. 
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The primary case of interest for this article is analytic diffeomorphisms, but we 
focus on groups with full support of finite type to emphasize the properties we use, 
which are dynamical rather than analytic in nature. The following result shows we 
include Diff 0"^(M). 

Proposition 1.2. If N is a compact connected manifold, the group Diff'^(A'") has full 
support of finite type. 

Proof. If / is analytic and non-trivial the set Fix(/) is an analytic set in N and has 
no interior and the set N \ Fix(/) is a semianalytic set. Hence Fix(/) has finitely 
many components and A^\Fix(/) has finitely many components (see Corollary 2.7 of 
[3] for both these facts). □ 

The following result is due to Katok [11] who stated it only in the analytic case. 
For completeness we give a proof in section 3 but the proof we give is essentially the 
same as the analytic case. 

Proposition 1.3 (Katok [11]). Suppose G is a subgroup o/Diff^(M) which has full 
support of finite type and f e Diff^(M) has positive topological entropy. Then the 
centralizer of f in G, is virtually cyclic. Moreover, every infinite order element of 
this centralizer has positive topological entropy. 

A corollary of the proof of this result is the following. 

Corollary 1.4. Suppose f,g E DifT^(M) have infinite order and full support of finite 
type. If fg = gf, then f has positive topological entropy if and only if g has positive 
entropy. 

We remark that in contrast to our subsequent results Proposition 1.3 and its 
corollary make no assumption about preservation of a measure /i. 

We can now state our main result, whose proof is given in section 6. 

Theorem 1.5. Suppose M is a compact oriented surface of genus and G is a 
subgroup of Symp^(M) which has full support of finite type, e.g. a subgroup of 
Symp|;^(M). Suppose further that G has an infinite normal solvable subgroup. Then 
G is virtually abelian. 

There are several interesting corollaries of this result. To motivate the first we 
recall the following result. 

Theorem 1.6 (Farb-Shalen [5]). Suppose f G Diffo'^(5'^) has infinite order. Then 
the centralizer of f in Diffo'^(S'^), is virtually abelian. 

There are a number of instances where results about the algebraic structure of 
DifT(S'^) have close parallels for Symp^(M). By analogy with the result of Farb and 
Shalen above, it is natural to ask the following question. 



2 



Question 1.7. Suppose M is a compact surface and G is a subgroup o/Diffo^(M) 
with full support of finite type and f & G has infinite order. Is the centralizer of f in 
G always virtually abelian? 

The following result gives a partial answer to this question. 

Proposition 1.8. Suppose M is a compact surface of genus 0, H is a subgroup 
o/ Symp^(M) with full support of finite type and f & H has infinite order, then 
Cent^(/, if), the centralizer of f in H, is virtually abelian. In particular this applies 
when H C Symp'^{M). 

Proof. If we apply Theorem 1.5 to the group G — Cent^(/, H) and observe that the 
cyclic group generated by / is a normal abelian subgroup, then we conclude that G 
is virtually abelian. □ 

Question 1.9 of [6] asks if any finite index subgroup of MCG(Sc,) can act faithfully 
by area preserving diffeomorphisms on a closed surface. The following corollary of 
Proposition 1.8 and Proposition 1.2 gives a partial answer in the special case of 
analytic diffeomorphisms and M of genus 0. 

Corollary 1.9. Suppose H is a finite index subgroup of MCG(Eg) with g > 2 or 
Out(F„) with n > 3, and G is a group with the property that every element of infinite 
order has a virtually abelian centralizer. Then any homomorphism v : H ^ G has 
non-trivial kernel. In particular this holds if G — Symp^(S'^). 

Proof. Suppose at first that if is a finite index subgroup of MCG(Sj) with g > 2. 
Let ai, a2 and be simple closed curves in M such that cti and a2 are disjoint from 
0:3 and intersect each other transversely in exactly one point. Let be a Dehn twist 
around with degree chosen so that Tj e H. Then Ti and T2 commute with T3 but 
no finite index subgroup of the group generated by Ti and T2 is abelian. It follows 
that 11 cannot be injective. 

Suppose now that if is a finite index subgroup of Out(F„) with n > 3. Let A, B 
and G be three of the generators of F„. Define $ e Aut(F„) by B BA, define 
^ G Aut(F„) hyC ^ CBAB-^A and define O G Aut(F„) by C ^ CABAB'^, where 
$ fixes all generators other than B and where ^ and fix all generators other than C . 
Let 0, %Ij and be the elements of Out(F„) determined by $, ^' and respectively. It 
is straightforward to check that $ commutes with ^ and and that for all A; > 0, ^''^ 
(which is defined by C i-> G{BAB^^AY) does not commute with 0^^ (which is defined 
by C I— )■ G{ABAB~^Y). Moreover, [0^, ^'^j is not a non-trivial inner automorphism 
because it fixes both A and B. It follows that [6^, 'ijj'^] is non-trivial for all k and 
hence that no finite index subgroup of the group generated by ip and 6*, and hence 
no finite index subgroup of Cent(0), the centrahzer of in Out(F„), is abehan. The 
proof now concludes as above. □ 

Another interesting consequence of Theorem 1.5 is the following result. 
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Proposition 1.10. Suppose M is a compact surface of genus and G is a solvable 
subgroup o/Symp^(M) with full support of finite type, then G is virtually abelian. 

The Tits alternative (see J. Tits [15]) is satisfied by a group G if every subgroup (or 
by some definitions every finitely generated subgroup) of G is eitlier virtually solvable 
or contains a non-abelian free group. This is a deep property known for finitely 
generated linear groups, mapping class groups [9], [12], and the outer automorphism 
group of a free group [1] [2]. It is an important open question for Diff'^(5'^) (see [5]). 
It is known that this property is not satisfied for Diff°°(5'^) (again see [5]). 

This naturally raises the question for surfaces. 

Conjecture 1.11 (Tits alternative). If M is a compact surface then every finitely 
generated subgroup of Symp^(M) is either virtually solvable or contains a non-abelian 
free group. 

In the final section of this paper we are able to use the techniques developed to 
prove an important special case of this conjecture. 

Theorem 1.12. Suppose that M is a compact oriented genus zero surface, that G is 
a subgroup o/Symp||^(M) and that G contains an infinite order element with entropy 
zero and at least three periodic points. Then either G contains a subgroup isomorphic 
to F2, the free group on two generators, or G has an abelian subgroup of finite index. 

We observe that one cannot expect the virtually abelian (as opposed to solvable) 
conclusion to hold more generally as the subgroup of Symp|;^(T^) generated by an 
ergodic translation and the automorphism with matrix 



2 Maximal Annuli for elements of Symp°°(/S^). 



Definition 2.1. Suppose M is a compact oriented surface. The annular compactifi- 
cation of an open annulus ?7 C M is obtained by blowup on an end whose frontier is 
a single point and by the prime end compactification otherwise. We will denote it by 
Uc (see 2.7 [6] for details). 

For any diffeomorphism h oi M which leaves U invariant there is a canonical 
extension to a homeomorphism he : Uc ^ Uc which is functorial in the sense that 
{hg)c = hcgc- 

Definition 2.2. Suppose M is a compact genus zero surface, / G Symp^(M), and 
that the number of periodic points of / is greater than the Euler characteristic of M. 
If / has infinite order and entropy 0, we will call it a multi-rotational diffeomorphism. 
This set of diffeomorphisms will be denoted Z[M). 




is solvable, but not virtually abelian. 
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The rationale for the terminology multi-rotational comes from the following result 
which is a distillation of several results from [6] (see Theorems 1.2, 1.4 and 1.5 from 
that paper). In particular, if / G Z{M) then every point of int(M) \Fix(/) has a well 
defined rotation number (with respect to any pair of components from dM U Fix(/) 
and there are non-trivial intervals of rotation numbers. 

Theorem 2.3. Suppose M is a compact genus zero surface and f G Z{M). The 
collection A = Aj of maximal f -invariant open annuli in int(M) \ Fix(/) satisfies 
the following properties: 

(1) The elements of A are pairwise disjoint. 

(2) The union \^ U is a full measure open subset of \Fix(/). 

ueA 

(3) Each U e A is essential in int(M) \ Fix(/). 

(4) For each U E A, the rotation number pf : Uc ^ is continuous and non- 
constant. Each component of the level set of pf which is disjoint from dUc 
is essential in U, i.e. separates Uc into two components, each containing a 
component of dUc. 

We will make repeated use of the properties in the following straightforward corol- 
lary. 

CoroUciry 2.4. Suppose M is a compact genus zero surface, f G Z{M) and Cent^(/) 
denotes its centralizer in Symp^(M). 

(1) Ifge Cenf^if) and U e Af then g{U) G Af and the Cenf^ {f)-orbtt of U is 
finite. 

(2) If U E Af then any g G Ccnt^(/) which satisfies g{U) = U must preserve the 
components of each level set of pj : Uc ^ . 

(3) If f has support of finite type then the set Af of maximal annuli for f is finite. 

Proof, li g e Cent^(/) then c/(Fix(/)) = Fix(/) so int(M) \ Fix(/) is c/-invariant. 
Also f{g{U)) = g{f{U)) = g{U) so g{U) is /-invariant. Clearly [/ is a maximal /- 
invariant annulus in int(M)\Fix(/) if and only if g{U) is. This proves g{U) G Af. The 
Cent^(/)-orbit of U consists of pairwise disjoint annuli each with the same positive 
measure. There can only be finitely many of them in M. This proves (1). 

To show (2) we observe that g is b, topological conjugacy from / to itself and 
rotation number is a conjugacy invariant. Hence g must permute the components of 
the level sets oi pf. Clearly those level sets which contain a component of dUc must 
be preserved. Since any other such component separates U , if it were not gf-invariant 
the fact that g is area preserving would be contradicted. 
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Since the elements of Af are maximal /-invariant annuli in int(M) \ Fix(/) , 
they are mutually non-parallel in int(M) \ Fix(/) . Let E be the union of a set of 
core curves for some given finite subset of Af. Theorem 2.3-(3) implies that each 
component oi \ E either contains a component of Fix(/) U dM or has negative 
Euler characteristic. (In fact a component with negative Euler characteristic would 
contain a point of Fix(/), but we will not need this fact.) The dual graph for E 
is a tree that has one edge for each element of E and has at most as many valence 
one and valence two vertices as there are components of Fix(/) U dM. Since the 
Euler characteristic of the tree is 1 there is a uniform bound on the number of its 
edges. It follows that the cardinality of E, and hence the number of elements of Af, 
is uniformly bounded. This proves (3). □ 



3 The positive entropy case. 

Lemma 3.1. Suppose that M is a closed surface, that f G Diff^(M) and that g 
commutes with f . Suppose further that f has a hyperbolic fixed point p of saddle 
type, and that g fixes p and preserves the branches of W'^{p, f). Then there is a 
coordinate function t on W^(p,f) and a unique number a > such that in these 
coordinates g{t) — at. In particular a is an eigenvalue of Dg^. 

Proof. By Sternberg hnearization (see Theorem 2 of [14]) there is a coordinate 
function t{x) on W^{p, f) in which the restriction of / (also denoted /) satisfies 
f{t) = Xt where A e (0, 1) is the eigenvalue of Dfp corresponding to the eigenvector 

tangent to W^{p, /). 

In these coordinates g{t) = X~"'g{\"'t). Applying the chain rule gives g'{t) = 
g'{X^t). Letting n tend to infinity we get g'{t) = g'{0) so g'{x) is constant and g{t) = at 
where a — g'{0). □ 

Definition 3.2. Suppose that M is a compact surface and that / e Diff''(M) has 
a hyperbohc fixed point p of saddle type. We define Centp(/) to be the subgroup 
of elements of Diff''(M) which commute with /, fix the point p, and preserve the 
branches of W^{f,p). Let M+ denote the multiphcative group of positive elements of 
M. The expansion factor homomorphism 

cj> : Cent;(/) ^ M+, 

is defined by (j){g) = a where a is the unique number given by Lemma (3.1) for 
which g{x) = ax in coordinates on W'^{f,p). It is immediate that (p is actually 
a homomorphism. We also observe that (f>{g) is just the eigenvalue of Dgp whose 
eigenvector is tangent to W^{f,p). 

The following result is due to Katok [11] who stated it only in the analytic case, 
but gave a proof very similar to the one below. 
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Proposition 1.3 ([Katok [11]]). Suppose G is a subgroup o/Diff^(M) which has full 
support of finite type and f E Diff^(M) has positive topological entropy. Then the 
centralizer of f in G, Ccnt^(/, G), is virtually cyclic. Moreover, every infinite order 
element 0/ Cent^(/, G) has positive topological entropy. 

Proof. By a result of Katok [10] there is a hyperbolic periodic point p for / of saddle 
type with a transverse homoclinic point q. Let : Centp(/) — )> IR+ be the expansion 
factor homomorphism of Definition 3.2 and let H = Centp(/) HG. Then H has finite 
index in Cent^(/, G) because otherwise the G orbit of p is infinite and consists of 
hyperbolic fixed points of / all with the same eigenvalues. This is impossible because 
a limit point would be a non-isolated hyperbolic fixed point. For the main statement 
of the proposition, it suffices to show that H is cyclic and for this it suffices to show 
that the restriction (j)\H '■ H ^ M"*" is injective and has a discrete image. 

To show that (f)\H is injective it suffices to show that \i g E H and (t){g) — 1 then 
g = id. But (t){g) = 1 implies W'{p, f) C Fix(c/). Note that if 0' : CentJ(/-i) R+ 
is the expansion homomorphism for then (f)[g)(f)' {g) = det{Dgp) = 1 so we also 
know that W^{p, f) C Y\x{g). Hence we need only show that this implies g = id. Let 
Jg be the interval in W^{p,f) joining p to g and define J„ C W"{p,f) analogously. 
Define = f^"'{Js) U /"(<^«)- Then the number of components of the complement 
of Kn tends to +00 with n. Since g has full support of finite type, each component 
V of the complement of Kn contains a component of M \ Fix[g), in contradiction to 
the fact that M \ Fix(g') has only finitely many components. We conclude that is 
injective. 

To show that (pln has discrete image we assume that there is a sequence {gn} of 
elements of H such that lim0(g'„) = 1, and show that (f){gn) = 1 for n sufficiently 
large. Let Ig = f~^{Js) and = f{Ju)- Since Is and are compact intervals and 
the point g is a point in the interior of each where they intersect transversely, there 
is a neighborhood U of q such that f/ fl fl /„ = {q}. But for n sufficiently large 
gn{(l) G U and gn{q) G {Is ^ ^u)- It follows that gn{q) = q ioi n sufficiently large and 
hence (/)(fii„) = 1. This proves that the image of 0^ is discrete and hence that H is 
cyclic. 

Each non-trivial element h E H has 7^ 1 and (f)'{h) 7^ 1. Hence p is a 

hyperbolic fixed point of h and g is a transverse homoclinic point for h. It follows 
that h has positive entropy. □ 

Proposition 3.3. Suppose G is a subgroup ci/Diff^(M^) which has full support of 
finite type and A is an abelian normal subgroup of G. If there is an element f E A 
with positive topological entropy then G is virtually cyclic. 

Proof. It follows from Proposition 1.3 that the group A is virtually cyclic. Since / E 
has positive entropy it is infinite order and generates a finite index subgroup Af of A. 
Hence there exists a positive integer k such that a'^ E Af for all a E A. In particular, 
for each g E G, we have gf^g~^ — {gfg~^)^ — for some m e Z . Since / and 
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gfg ^ are conjugate, the topological entropy ent(/) = ent{gfg Hence 

entigfg-')' = \k\ent{gfg-') = |^|ent(/), and ent(r) = |m|ent(/). 

We conclude that m — ±/c and hence that gf^'g^^ = /^*^ for all g E G. Let Go be 
the subgroup of index at most two of G such that gf^g~^ = for all g e Gq. Then 
Go C Cent^(/'^,G) and Proposition 1.3 completes the proof. □ 

4 Mean Rotation Numbers 

In this section we record some facts (mostly well known) about rotation numbers 
for homeomorphisms of area preserving homeomorphisms of the closed annulus. In 
subsequent sections we will want to apply these results when M is a surface, / e 
Symp^(M), and U is an /-invariant open annulus. We will do this by considering 
the extension of / to the closed annulus Uc which is the annular compactification of 
U. When the annulus U is understood, we will use pf{x) to mean the rotation number 
of X e [/ with respect to the homeomorphism fc'Uc^ Uc of the closed annulus Uc- 

Definition 4.1. Suppose that / : A ^ A is a homeomorphism of a closed annulus 
A = 5'-'^ X [0, 1] preserving a measure For each lift /, let Ay-(a;) = Pi{f{x)) — pi{x) 
where x is a hft of x and pi : ^4 = ^ x [0, 1] — > R is projection onto the first factor. 
As reflected in the notation, Aj(x) is independent of the choice of lift x and hence 

may be considered as being defined on A. 

If X C A is an /-invariant yU-measurable set then the mean translation number 
relative to X and / is defined to be 

%{f,X)= [ Aj{x)di,. 
Jx 

We define the mean rotation number relative to X, Pix{f, X) to be the coset of 7^(/, X) 
mod Z thought of as an element of T = R/Z. 

The mean rotation number is independent of the choice of lift / since different 
hfts give values of T(/, X) differing by an element of Z. 

We define the translation number oi x with respect to / (see e.g. Definition 2.1 of 
[6]), by ^ 

rf{x) — lim —Afn{x). 

■' n—>oo n ■' 

A straightforward application of the Birkhoff ergodic theorem implies that tj{x) is 
well defined for almost all x and 

%{lX)= f Tj{x)dp. 

Jx 
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If X is also gi-invariant and g preserves // then 

Aj-{x) = Ag{x) + Aj{~g{xj) 

and hence integrating we obtain 

%{f~g, X) = %{f, X) + %{~g, X) and p,{fg, X) = p,{f, X) + p,{g, X), 

I.e. / I—)- Pn{f,X) is a homomorphism from the group of p preserving homeomor- 
phisms of A to T = M/Z. Hence ii h = [gi,g2] for some gfj : A ^ A that preserve p 
then p^{h) = 0. 

Lemma 4.2. Let f : A ^ A be an area preserving homeomorphism of a closed an- 
nulus isotopic to the identity. Suppose X G A is an f -invariant subset with Lebesgue 
measure p{X) > 0. If p^{f,X) — then f has a fixed point in the interior of A. 

Proof. Replacing X with X fl int(74) we may assume X C int(74). If tj{x) = on 
a positive measure subset of X then the fixed point exists by Proposition 2.4 of [6]. 
Otherwise for some e > there is a positive measure /-invariant subset X^ on which 
Pf > e and another X^ on which p/ < — e. If x G X^ is recurrent and not fixed 
then it is contained in a positively recurring disk which is disjoint from its /-image. 
Similarly if y G X~ is recurrent and not fixed then it is contained in a negatively 
recurring disk which is disjoint from its /-image. Theorem 2.1 of [7] applied to / on 
the open annulus int(>l) then implies there is a fixed point in int(A). □ 

Lemma 4.3. Suppose that f : A ^ A is an area preserving homeomorphism of a 
closed annulus which is is isotopic to the identity. Suppose also that V (Z A is a 
connected open set which is inessential in A and such that f"^{V) = V for some 
m 7^ 0. Then there is a full measure subset W ofV on which pf assumes a constant 
rational value. 

Proof. Replacing V with V fl int(A) we may assume V C int(A). Replacing / with 
/"* it will suffice to show that f{V) — V implies p/ = on a full measure subset of 
V. Since V does not contain a simple closed curve that separates the components of 
dA, both components of dA belong to the same component X of A \ \/ by Lemma 
3.1 of [6]. Note that D := A\X contains V, is contained in int(A) and is /-invariant. 
Lemma 3.2 of [6] implies that D is connected and it is simply connected since its 
complement X is connected. Hence D is an open disk. 

By the Brouwer plane translation theorem D contains a point of Fix(/). Let A 
be the universal cover of A and let D G A be an open disk which is a lift of D. Since 
/ has a fixed point in D there is a lift f : A ^ A which has a fixed point in D. 
Therefore f{D) ~ D. If pi{D) is bounded then it is obvious that p/ is constant and 
on D. Otherwise, note that by Poincare recurrence and the Birkhoff ergodic theorem 
there is a full measure subset W oi D consisting of points which are recurrent and 
have a well defined translation number. Calculating the translation number of a point 
X & W on a. subsequence of iterates which converges to a point of A shows that it 
must be 0. Hence W , the projection of W to D, has the desired properties. □ 



9 



5 Pseudo-rotation subgroups of SympJ^(M). 

Definition 5.1. Suppose M is a compact oriented surface. A pseudo-rotation sub- 
group of Symp|[^(M) witli r > 1, is a subgroup G witli tlie property tliat every non- 
trivial element of G has exactly x{M) fixed points. An element / e SympJ^(M) will 
be called a pseudo-rotation provided the cyclic group it generates is a pseudo-rotation 
group. 

Our definition is slightly non-standard in that we consider finite order elements of 
Symp^(M) to be pseudo-rotations. We observe that if x(M) < then any pseudo- 
rotation group is trivial. Since we assume M is oriented we have only the cases that 
M is A, D^, or S*^ for which any pseudo-rotations must have precisely 0, 1, or 2 
fixed points respectively. By far the most interesting case is M = 5*^, since we will 
show in Lemma 5.7 that when M = or any pseudo-rotation group is abelian. 
This is not the case when M — S"^ since SO{3) acting on the unit sphere in is a 
pseudo-rotation group. 

There are three immediate but very useful facts about pseudo-rotations which we 
summarize in the following Lemma: 

Lemma 5.2. Suppose M is a compact surface and f e Symp|[^(M) is a pseudo- 
rotation. 

(1) Either f has finite order or Fix(/) = Per(/). 

(2) //Fix(/) contains more than x(M) points then f — id. In particular if f,g are 
elements of a pseudo-rotation group which agree at more than x(M) points then 
fg-^ ^id so f ^g. 

(3) If M = D^, then the one fixed point of f is in the interior o/D^. 

Proof. Parts (1) and (2) are immediate from the definition of pseudo-rotation and part 
(3) holds because otherwise the restriction of / to the interior would have recurrent 
points but no fixed point, contradicting the Brouwer plane translation theorem. □ 

We will make repeated use of these properties. 

Proposition 5.3. Suppose f e Symp^(M) is a pseudo-rotation. Then U = int(M) \ 
Per(/) is an open annulus and pf : Uc ^ T is a constant function. Moreover, if 
G C Symp^(M) is a pseudo-rotation subgroup, the assignment f i-^ pf, for f & G, is 
an injective homomorphism, so, in particular, if f has infinite order then the constant 
Pf is irrational. 

Proof. The only possibilities for M are 5^, A or D^. If M = D^, as remarked above, its 
one fixed point must be in the interior of M. Hence in all cases U is an open annulus. 
If p/ is not constant on Uc then the Poincare-Birkhoff Theorem (see Theorem (2.2) 
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of [6] for example) implies there are periodic points with arbitrarily large period. We 
may therefore conclude that p/ is constant and equal to Suppose now that G 

is a pseudo-rotation subgroup. Since : G — > T is a homomorphism p^{f) = Pfj,{g) 
implies Pn{fg~^) = 0. Then Lemma 4.2 implies the existence of a fixed point in U 
for fg~^. Hence fg~^ — id and the assignment / P^i{f ) is injective. We conclude 
that if / has infinite order then is irrational. □ 

We observed above that if / is a non-trivial pseudo-rotation then either / has 
finite order or Per(/) = Fix(/). We now prove the converse to this statement. 

Proposition 5.4. Suppose M is A, or S"^ andG is a subgroup o/Symp^(M) with 
the property that every non-trivial element g of G either has finite order or satisfies 
Fix{g) — Per{g). Then G is a pseudo-rotation group. 

Proof. If gi e G, then it must have at least x(^) fixed points because the Lefschetz 
index of a fixed point of g is at most 1 (see [13], for example). If g is non-trivial 
and has more than x(M) fixed points it follows from part (4) of Theorem 2.3 that 
there is an 51-invariant annulus U G J (Af ) for which the rotation number function 
in continuous and non-constant. It then follows from Corollary 2.4 of [8] that there 
are periodic points with arbitrarily high period, contradicting our hypothesis. We 
conclude that either g has finite order or card(Per((7)) = xi^). □ 

Lemma 5.5. Suppose U G M is an open annulus and that Ci and C2 are disjoint 
closed connected sets in U each of which separates the ends of U and such that both 
U \Ci and U\C2 have two components. Then M \ {Ci U C2) has three components: 
one with frontier contained in Ci, one with frontier contained in C2 and an open 
annulus with frontier contained in (Ci U C2) . 

Proof. Clearly M \ Ci has two components. Let Xi and X2 [resp. Yi and ¥2] be 
the components of M \ Ci [resp. M \ C2] labeled so that C2 C X2 and Ci C Yi. 
Then Xi and Y2 are components of M \ (Ci U C2) with frontiers contained in Ci 
and C2 respectively and it suffices to show that V = X2 fl Yi is an open annulus. 
We use the fact that an open connected subset of whose complement has two 
components is an annulus. But XiU Ci — cl(Xi) U Ci is connected and similarly so 
is Y2U C2 — cl{Y2) U C2. Hence M \ V has two components, namely Xi U Ci and 
y2 U C2. SoV is an annulus. □ 

Our next result includes a very special case of Proposition (1.8) namely we show 
that the centralizer of an infinite order pseudo-rotation is virtually abelian. In fact, 
for later use, we need to consider a slightly more general setting, namely, not just 
the centralizer of a single pseudo-rotation, but the centralizer of an abelian pseudo- 
rotation group all of whose non-trivial elements have the same fixed point set. One 
notable thing about this special case is that we do not need the hypothesis of full 
support of finite type which will be necessary for the proof in more general cases. 
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Lemma 5.6. Suppose A is an abelian pseudo-rotation subgroup o/Symp^(M) con- 
taining elements of arbitrarily large order. Suppose also that there is a set F with 
points such that Fix(/) = F for all non-trivial f E A. Then the subgroup Cq 
of the centralizer, Cent^(74), of A in Symp^(M) consisting of those elements which 
pointwise fix F , is abelian and has index at most 2 in Cent^(A). 

Proof. Elements of A must have entropy since positive entropy implies the existence 
of infinitely many periodic points by a result of Katok, [10] . Let U be the open annulus 
M \ (F U dM). The group Cent^(^) preserves F U dM and hence U. The subgroup 
Co C Cent^(yl), whose elements fix the ends of U , has index at most 2. 

Wc claim that the elements of Co all have entropy 0. Clearly we need only consider 
elements of infinite order since all finite order homeomorphisms have entropy 0. Hence 
the claim follows from Corollary (1.4) if there is an element of infinite order in A. 
Otherwise if Co contains an element with positive entropy then it contains an element 
g with a hyperbolic fixed point p & U. Each point in the A-orbit of p is in Fix{g) 
and has the same set of eigenvalues for the derivative Dg. It follows that the A-orbit 
of p has finite cardinality, say m, and hence that p G Fix(/™) for all f & A. Part 
(2) of Lemma 5.2 then implies that f"^ — id for all f ^ A. This contradicts the fact 
that A contains elements of arbitrarily high order. This completes the proof that all 
elements of Co have entropy zero. 

To prove that Co is abelian, we will show that each commutator h of two elements 
in Co is the identity by assuming that h is non-trivial and arguing to a contradiction. 
Since h is a commutator the map he : Uc ^ Uc has mean rotation number p^ihc) = 
and hence, by Lemma 4.2, h has a fixed point in U. Therefore Fix(/i) contains 
more than xi^) points. If h has finite order then in a suitable averaged metric it 
is an isometry of M. But then each fixed point must have Lefschetz number +1 
and hence by the Lefschetz theorem h has x{^) fixed points, a contradiction. Wc 
conclude therefore that h has infinite order and hence satisfies the hypothesis of 
Theorem (2.3). By this theorem there exists an element V e Ah and it must be the 
case that V G U since V cannot contain a point of Fix(^) D F. Wc will show this 
leads to a contradiction, either by showing that h has a fixed point in V or by showing 
that some non-trivial f E A has a fixed point in U . We may then conclude h = id 
and hence that Co is abelian. 

Suppose first that V is inessential in U. Elements of A commute with h and hence 
permute the elements of Ah by Corollary 2.4. Since there can be only finitely many 
elements of Ah of any fixed area, the A-orbit of V is finite. Hence there is m such 
that f"^{V) = V for all f E A. The union of V with the component of its complement 
which is disjoint from F is an open disk D G U satisfying f^[D) = D for all / G A. 
It follows from the Brouwer plane translation theorem that has a fixed point in 
D. If the order of / is greater than m then is a non-trivial element of A with 
fixed points not in F contradicting the assumption that Fix(/) = F for all non-trivial 
f E A. So the possibility that V is inessential in U is contradicted. 

We are now reduced to the case that V is essential in U. It follows from part (1) 
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of Corollary 2.4 that each element of A preserves V or maps it to a disjoint element 
of Ah- Since the annulus U is A-invariant and V is essential in U it follows that V 
is A-invariant as the alternative would contradict the fact that A preserves area. We 
want to replace V with a slightly smaller essential Vq G V which has the property 
that its frontier (in M) lies in V and has measure 0. To do this we observe that ph 
is non-constant on V and hence has uncountably many level sets. Hence there must 
be two of its level sets Ci,C2 which have measure 0. Let Vq be the essential open 
annulus in V whose frontier lies in C1UC2 (see Lemma 5.5). Then /i(Vo) = /i(clc/^(Vo)) 
since cl[/^(Vo) \ Vq C Ci U C2. It follows from part (2) of Corollary 2.4 that each Cj is 
>l-invariant and hence Vo is also. 

As a first subcase, suppose that clu^{Vo) is gr- invariant for each g E Co- Then h is 
a commutator of elements that preserve cl;7^(Vo), so 

(measured in Uc). Since Vq differs from cl;/^(Vo) in a set of measure zero, Pfj,{h, Vq) = 
also. The translation number with respect to a lift /i of a point x E Vq can be measured 
in either Uc or (Vo)c giving tj^{x) and , (x). But x E Vq lies on a compact /i-invariant 

level set Co which is in the interior of both Uc and (Vo)c- This implies tj^{x) = rj^, , (x). 

Hence we conclude that Pju(/i) = 0, measured in (Vo)c and by Lemma 4.2, h has a 
fixed point in Vq, a contradiction. 

The last subcase is that there exists g E Co such that g{Vo) ^ cIm{Vo) — clu^{Vo)- 
Choose a component W of g{Vo) (1 {U \ cl^^(Vo)). Since g leaves U invariant and 
preserves area and since g^^{W) (1 W = (l), W cannot contain a closed curve a that 
is essential in U. Thus W is inessential in U. As we noted above Vq is A-invariant. 
Since g E Co, it commutes with A so g{Vo) is also A-invariant. It follows that A 
permutes the components of ^'(^0) n (C/ \ clu^{Vo))- In particular for some m > we 
have f"^{W) = W for all f E A. Letting D be the union of W with any components 
of its complement which do not contain ends of U we conclude that D is an open disk 
and f"''{D) = D for all f E A. By the Brouwer plane translation theorem there is a 
point of Fix(/'") in D. Since is a pseudo-rotation and has more than x(M) fixed 
points, = id. Since this holds for any fEAwe have contradicted the hypothesis 
that A contains elements of arbitrarily high order. □ 

Lemma 5.7. Suppose that G is a pseudo-rotation subgroup o/Symp^(M) where M — 
A orB^ andr > 2. Then G is abelian and Fix{g) is the same for all non-trivial g E G. 

Proof. If M = A then Fix{g) = for all non-trivial g E G. Since Pfj.{[f,g]) = for 
each f,gEG, Lemma 4.2 implies that each [/, g] has a fixed point in the interior of 
A and hence is the identity. Thus each / and g commute and we arc done. 

We assume for the remainder of the proof that M = D^. For each f E G we 
consider /, the restriction of / to c?D^. Let G be the image in Diff^(5'^) of G under 
the homomorpliism / 1— )■ /. 
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Lemma 5.2 (3) implies that / fixes a point in the interior of D^. If f'' fixes a 
point in (9D^ for some A; > 1 then /^^ has more than x{^) fixed points and so is the 
identity. We conclude that if / has a point of period k then / is periodic of period 
k. In particular, if f : ^ has a fixed point then / = id. This also proves that 
f ^ f defines an isomorphism G ^ G. 

We will prove that G is abelian by showing that G is abehan. If each element of G 
has finite order then G acts freely on and Holder's Theorem (see, e.g.. Theorem 2.3 
of [5]) implies that G is abelian. We may therefore assume that some / has infinite 
order. In this case / has an irrational rotation number and is topologically conjugate 
to an irrational rotation by Denjoy's Theorem. Use this conjugacy to define a metric 
distance function on in which / is an isometry. If e G we claim that g is also 
an isometry with respect to this metric. Since the group of orientation preserving 
isometrics of the circle is abelian, this will prove that G is abelian as desired. 

If g is not an isometry, there exists an interval J whose length is strictly decreased 
by g. Since / is an irrational rotation there exists k such that j{g{J)) C J. It follows 
that f^g has a fixed point in J and hence that f^g is the identity. This contradicts 
the assumption that g is not an isometry with respect to the given metric. This 
completes the proof of the claim and hence the proof that G, and hence G. is abelian. 

Suppose f,g & G and let Fix(/) = {q}. Since / and g commute, g preserves {q} 
and so fixes q. This proves that Fix(/) = Fix{g) and completes the proof. □ 

If / e Diff'"(5'^) and p e Fix(/) we can compactify S'^ \ {p} by blowing up 
p, i.e. adding a boundary component on which we extend / by letting it be the 
projectivization of Dfp. More precisely we define fp : ^ by considering the 
boundary as the unit circle in and letting 

^'^^ \DfM\- 

If Fix(/) contains two points then we may blow up these points and obtain the 
annular compactification A of 5"^ \ Fix(/). 

Corollary 5.8. Suppose that G is a pseudo-rotation subgroup o/Symp^(S'^) and that 
Gp is the stabilizer of a point p E S^. Then Gp is abelian and there exists q & such 
that Fix(5f) = {p, q} for all non-trivial g G Gp. 

Proof. Suppose that / G Gp. Blowing up the point p and extending / we obtain 
/ : — )■ D^. This construction is functorial in the sense that ii h = fg then h = fg. 
Hence the assignment / h-)- / is a homomorphism from Gp to Symp^(B^). This 
homomorphism is injective by part (2) of Lemma 5.2. The result now follows from 
Lemma 5.7. □ 

It is not quite the case that for abelian pseudo-rotation subgroups of Symp^(5'^) 
the fixed point set for non-trivial elements is independent of the element. There 
is essentially one exception, namely the group generated by rotations of the sphere 
through angle n about two perpendicular axes. 
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Proposition 5.9. If G is an abelian pseudo-rotation subgroup of Symp^(5'^) then 

either Fix{g) is the same for every non-trivial element g of G or G is isomorphic to 
Z/2Z © Z/2Z. In the latter case the fixed point sets of non-trivial elements of G are 
pairwise disjoint and hence their union contains six points. 

Proof. Let g,h be distinct non-trivial elements of G and suppose Fix(5f) ^ Fix{h). 
Since G is abelian g(Fix{h)) = Fix[h). Since Fix(5f) ^ Fix{h), these sets are disjoint 
and each contains two points. Hence g switches the two points of Fix{h). In this case 
g has two fixed points and a period 2 orbit and hence g'^ has four fixed points and 
must be the identity. We conclude that g has order two. Switching the roles of g and 
h we observe that h has order two. 

Let Go be the abelian group generated by g and /i so Gq — ^/2Z © Z/2Z. The 
possible actions of an element of Go on Fix{g) U Fix(/i) are: fix all four points, switch 
the points of one pair and fix the other, or switch both pairs. If / e G then / preserves 
Fix(g') and Fix(/i) so its action on Fix{g) U Fix(/i) must be the same as one of the 
elements of Gq. Since / agrees with an element of Gq at four points it must, in fact, 
be an element of Gq. Hence any abelian pseudo-rotation group containing non-trivial 
elements whose fixed point sets do not coincide must be isomorphic to Z/2Z©Z/2Z. 

Suppose now Go = Z/2Z ® Z/2Z is a pseudo-rotation subgroup of Symp^(5'^). 
The three non-trivial elements of Go must have pairwise disjoint fixed point sets. To 
see this note that otherwise there would be two elements, say, g and h with a common 
fixed point. Blowing up that fixed point and restricting to the boundary would give 
an action of Go on each element of which is orientation preserving. Such an action 
cannot be faithful since it is conjugate to an action by isometries. It follows from 
part (2) of Lemma 5.2 that the action of Go on S'^ is not faithful. This contradiction 
implies the fixed points of the three non-trivial elements of Go are distinct proving 
the final assertion of the theorem. □ 

Lemma 5.10. Suppose G is a pseudo-rotation subgroup of Symp^(5'^) such that 
either 

(1) G leaves invariant a finite set F (Z S^, or 

(2) G has a non-trivial normal solvable subgroup. 

Then either G is finite, or G has an abelian subgroup of index at most 2. Moreover, 
if the set F in (1) is pointwise fixed then G is abelian. 

Proof. Suppose (2) holds. We will reduce this to the case that (1) holds. If N is 
a non-trivial normal solvable subgroup, let A be the last non-trivial element of its 
derived series so A is an abelian normal subgroup. By Proposition 5.9 the set 

F = y Fix{h) 

heA,h^id 
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is finite, li g E G and h E A then ghg~^ e A. Since Fix{ghg~^) — g{Fix.(h)), it follows 
that g{F) — F. Hence we need only prove the conclusion under the assumption that 
G leaves invariant a finite set F. 

Let Go be the finite index subgroup of G which pointwise fixes F. If F contains 
more than two points then Gq must be trivial and G — G/Gqis finite. If F contains 
two points then Go has index at most two and Fix(/i) = Fix(/i') for all h, h! e Gq. 
Then Go is abehan by Corollary 5.8. □ 

Proposition 5.11. If G is a subgroup o/Symp^(Af) which has an infinite normal 
abelian subgroup A which is a pseudo-rotation group then G is virtually abelian. 

Proof. Since A is non-trivial, M is A, D^, or 5*^. Since A is infinite, Proposition 5.7 
and Proposition 5.9 imply there is a set F containing x{M) < 2 points such that 
F = Fix{h) for all h & A. Let U = M \ F. Observe that in all cases U is an annulus. 
The set F must be invariant under G since F = Fix{ghg^^) = g{Fix{h)) = g{F) for 
every element g in G. The subgroup Gq of G that pointwise fixes F has index at most 
two. Also elements of Gq leave U invariant and their restrictions to U are isotopic to 
the identity. 

By Lemma 5.3 the homomorphism : ^4 — >■ T-*^ given by / i->- is injective, 

where p^(/) is the mean rotation number on U^- Since conjugating h by an element 
g E Go does not change its mean rotation number we conclude that h = ghg~^ for all 
h E A. This means that Go is contained in Cent^(y4, G), the centralizer of A in G. 

If A contains elements of arbitrarily large order then the result follows from 
Lemma 5.6, so we may assume the order of elements in A is bounded. Since the 
order of elements of A is bounded there are only finitely many possible values for 
Pfiif) with f E A. Hence since the assignment / h-)- p,jt{f) is injective we may con- 
clude A is finite in contradiction to our hypotheses. 

□ 

Example 5.12. Let A be the subgroup of all rational rotations around the 2;-axis 
and let G be the Z/2Z extension of A obtained by adding an involution which rotates 
around the x-axis and reverses the orientation of the z-axis. Then every element of 
A has finite order while A itself has infinite order. Moreover, G is not abelian even 
though it has an index two abelian subgroup. 

6 Proof of the Main Theorem 

In this section we prove 

Theorem (1.5) Suppose M is a compact oriented surface of genus and G is a 
subgroup of Symp^(M) which has full support of finite type, e.g. a subgroup of 
Symp^(M). Suppose further that G has an infinite normal solvable subgroup. Then 
G is virtually abelian. 
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Throughout this section M will denote a compact oriented surface of genus 0, per- 
haps with boundary. We begin with a lemma that allows us to replace the hypothesis 
that G has an infinite normal solvable subgroup with the hypothesis that G has an 
infinite normal abelian subgroup. 

Lemma 6.1. If G is a subgroup c)/Symp^(M), which contains an infinite normal 
solvable subgroup, then G contains a finite index subgroup which has an infinite normal 
abelian subgroup. 

Proof. Let be the infinite normal solvable subgroup. The proof is by induction on 
the length k of the derived series of A^. If A; = then N is abelian and the result 
holds. Assume the result holds for k < ko for some ko > and suppose the length 
of the derived series for A^ is A; = /cq + 1- Let A be the abelian group which is the 
last non-trivial term in the derived series of A^. The group A is invariant under any 
automorphism of A^ and hence is normal in G. If A is infinite we are done. 

We may therefore assume A is finite and hence that in a suitable metric A is a 
group of isometrics. No non-trivial orientation preserving isometry of M can have 
infinitely many fixed points so we let F be the finite set of fixed points of non-trivial 
elements of A. Since A is normal g{F) — F for any g E G. Let Go be the finite 
index subgroup of G which pointwise fixes F. The intersection of Gq with its set of 
conjugates by a finite set of representatives from the left cosets of Gq is a normal 
subgroup Gi of finite index. 

Let A^i — Gi f] N so Ni is an infinite solvable normal subgroup of Gi and the 
k-ih term Ai in the derived series of A^i is contained in AoGi. We claim that Ai 
is trivial. If F contains more than x(Af) points then this follows from the fact that 
a non-trivial isometry isotopic to the identity fixes exactly x(M) points since each 
fixed point must have Lefschetz index +1. 

Otherwise F contains x(M) points and we may blow up these points to form 
an annulus A on which A^i acts preserving each boundary component. Since each 
element of Ai is a commutator of elements in A^i, it acts on A with mean rotation 
number zero. Each element of Ai therefore contains a fixed point in the interior of 
A by Lemma 4.2. But a finite order isometry of the annulus which is isotopic to the 
identity and contains an interior fixed point must be the identity as every fixed point 
must have Lefschetz index +1 but the Euler characteristic of A is 0. This shows that 
Ai acts trivially on A and hence on M. This verifies the claim and hence the length 
of the derived series for Ni is at most kg. The inductive hypothesis completes the 
proof. □ 

The next lemma states the condition we use to prove that G is virtually abelian. 

Lemma 6.2. Suppose Gq is a subgroup o/Symp^(M) which has full support of finite 
type. Suppose further that there is an infinite family of disjoint Go-invariant open 
annuli. Then Gq is abelian. 
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Proof. We assume that [Go, Gq] contains a non-trivial element h and argue to a contra- 
diction. For each Go-invariant open annulus V, let Vc be its annular compactification 
and let /ic : K — )■ Vc be the extension of h\v (see Definition 2.1 or Definition 2.7 of [6] 
for details). Since /i is a commutator of elements of Go and since Gq extends to an 
action on Vc, he is a commutator and so has mean rotation number zero. Lemma 4.2 
therefore implies that Fix(/i) fl F 7^ 0. 

By assumption, Fix{h) does not contain any open set and so Fix{h\y) is a proper 
subset of V. We claim that h{V) U Fix{h\y) is not connected. To see this, let S 
be the end compactification of V obtained by adding two points, one for each end 
of V and let hs : S ^ S be the extension of h\v that fixes the two added points. 
If fr(y) U Fix(/i|y) is connected then each component 1^ of 5" \ Fi-x{h\s) is an open 
disk. A result of Brown and Kister [4] asserts W is /i^- invariant. However, then the 
Brouwer plane translation theorem would imply that h has a fixed point in W. This 
contradiction proves the claim. 

By passing to a subfamily of the Go-invariant annuli, we may assume that the 
following is either true for all V or is false for all V: some component of Fix(/i) 
intersects both components of the frontier of V. In the former case, the interior of 
each V contains a component of Fix(/i). In the latter case, no component of Fix{h) 
intersects more than two of the annuli in our infinite family. In both cases, Fix{h) has 
infinitely many components in contradiction to the assumptions that h is non-trivial 
and that Go has full support of finite type. □ 

We need an elementary topology result. 

Lemma 6.3. Suppose that C C int(Af) is a closed connected set with empty interior 
and with two complementary components Ui and 1/2- Then C = fr([/i) fl fr(?72) is a 
closed connected set with two complementary components, each of which has frontier 
C and is equal to the interior of its closure. 

Proof. To see that C separates Ui and U2, suppose that cr is a closed path in M \ G' 
with initial endpoint in Ui and terminal endpoint in 1/2. Then a fl fr(C/2) 7^ and we 
let (To be the shortest initial segment of a terminating at a point x G cr nfr(f/2)- Each 
y G o"o \ a; has a neighborhood that is disjoint from f/2. Since C has empty interior, 
every neighborhood of y must intersects Ui. It follows that every neighborhood of x 
must intersect Ui and hence that x E C This contradiction proves that G' separates 
Ui and U2. Since the union of Ui and U2 is dense in M, the components Vi and V2 of 
M\C' that contain them are the only components of M \ C. Every neighborhood 
of a point in G' intersects both Ui and U2 and so intersects both Vi and V2. Thus 
G' C fr(Vi) nfr(V2)- Since fr(\/i), fr(\/2) C G' we have that G' C fr(Vi) n fr(V2) C 
fr(yj) C G' and hence G' = fr{Vi) = fr{V2) which imphes that Vi and V2 are the 
interior of their closures. □ 

Notation 6.4. If G and G' are as in Lemma 6.3 then we say that G' is obtained from 
G by trimming. 
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Recall that if / e Z{M) and U e Af, then the rotation number function p — 
Pf\u '■ U ^ is well defined and continuous. A component of a point pre-image of 
p is a level set for p and is said to be irrational if its p-image is irrational and to be 
interior if it is disjoint from the frontier of U. 

Lemma 6.5. Suppose G is a subgroup ofSjmp^{M) containing an abelian normal 
subgroup A, that f G Z{M) lies in A and that U G Af. Suppose further that C[ 
and C'2 are obtained from nowhere dense irrational interior level sets Ci and C2 for 
P ~ Pf\u trimming. Letting Bi be the component of M\ {C[ U C'2) with frontier C[ 
and V be the component of M\ {C[ U C2), with frontier {C[ U C'^, assume that 

(1) ii{V) < ii{B,) < pi{B2). 

(2) pi{B2) > pi{M)/2. 

Then for all g & G, either g{V) nV — or g{V) — V. In particular, there is a finite 
index subgroup of G that preserves V. 

Proof. We assume that giV) fl F 7^ and g{V) ^ V and argue to a contradiction. 
Since A is normal and abelian, h = g~^fg commutes with /. Corollary 2.4 part (1) 
implies that h permutes the elements of ^/ and hence that = U for some 

n > 1. It then follows from Corollary 2.4 part (2) that /i" preserves each level set 
ior p — pf\j^, and hence each trimmed level set for p, and so preserves V, Bi and B2. 
Equivalently, g{V). g{Bi) and (?(i?2) ai'c /"-invariant. 

Since g{V) fl V 7^ and giV) ^ V, there is a component W of g{V) fl V such 
that fr(l^) n fr(V) 7^ 0. To see this observe that otherwise one of V and g{V) would 
properly contain the other which would contradict the fact that g preserves p. Since 
/" preserves both V and g{V), it permutes the components of their intersection. 
Since / preserves area, /"'(VF) = W for some m > 0. If every simple closed curve in 
W is inessential in V then p has a constant rational value on a dense subset of W 
by Lemma 4.3. This contradicts the fact that p is continuous on U and takes only 
irrational values on b{V). We conclude that there is a simple closed curve a G W 
which is essential in V. Let /3 = g~^{a) C V. Item (1) implies that /3 is also essential 
in V because if it were inessential the component of its complement which lies in V 
would contain either g~^{Bi) or g~^{B2). 

Let B[ be the subsurface of M bounded by /3 that contains B^. Item (2) rules out 
the possibility that B2 C g{B[) so it must be that Bi C g{B'^) and B2 C g{B'2). Item 
(1) therefore implies that Bing{Bi) 7^ 0. If there is a simple closed curve in Bi whose 
gi-image is in V and essential in V then there is a proper subsurface of Bi whose g 
image contains Bi in contradiction to the fact that g preserves area. It follows that 
either g{Bi) = Bi or there is a component of g{Bi) fl V which is inessential in 
V and whose frontier intersects Q'. As above, this implies that p is constant and 
rational on W in contradiction to the fact that p is continuous on U and takes an 
irrational value on C^. This contradiction completes the proof. □ 
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We are now prepared to complete the proof of our main theorem. 

Theorem (1.5) Suppose M is a compact oriented surface of genus and G is a 
subgroup of Symp^(M) which has full support of finite type, e.g. a subgroup of 
Symp^(M). Suppose further that G has an infinite normal solvable subgroup. Then 
G is virtually abelian. 

Proof. By Lemma 6.1 it suffices to prove the result when G has an infinite normal 
abelian subgroup A. li A contains an element of positive entropy the result follows 
by Proposition 3.3. If the group A is a, pseudo-rotation group the result follows from 
Proposition 5.11. 

We are left with the case that A contains an element / G Z[M). Let U be an 
element of ^/ and let p — pf\^. Since there are only countably many level sets of 
p with positive measure, all but countably many have empty interior or equivalently 
are nowhere dense. Choose a nowhere dense interior irrational level set C of p. One 
component of M \C, say, Y, will be an open subsurface with ^{Y) < fi{M)/2. 
Choose a nowhere dense interior irrational level set C' G Y H U and let Y' be the 
open subsurface which is the component of the complement of G' satisfying Y' cY. 
Finally, choose nowhere dense interior irrational level sets Ci,C2 d {Y \ cl(y)) so 
that the annulus V bounded by the trimmed sets C[ and C2 has measure less than 
the measure of Y'. Then C[ and C2 satisfy the hypotheses of Lemma 6.5. It follows 
that the subgroup Go of G that preserves V has finite index in G. 

Note that any two trimmed irrational level sets G[ and C2 in V satisfy the hy- 
potheses of Lemma 6.5. Moreover if V is the essential open subannulus bounded by 
C[ and 6*2 then g{V) fl V 7^ for each g & Gq because each such g preserves V 
and preserves area. Lemma 6.5 therefore implies that each such V is Go-invariant so 
Lemma 6.2 completes the proof. □ 



7 The Tits Alternative 

We assume throughout this section that M be a compact oriented surface of genus 0, 
ultimately proving a special case (Theorem 1.12) of the Tits Alternative. 

Lemma 7.1. Suppose f e Z{M) and U e A{f) is a maximal annulus for f. If G is 

a subgroup o/Symp||^(M) whose elements preserve each element of an infinite family 
of trimmed level sets for pf\^ lying in U , then G is abelian. 

Proof. Since infinitely many of the trimmed level sets in U are preserved by G so 
is each open annulus bounded by two such level sets. It is thus clear that one may 
choose infinitely many disjoint open annuli in U each of which is G- invariant. The 
result now follows from Lemma 6.2. □ 

Lemma 7.2. Suppose f : A ^ A is a homeomorphism isotopic to the identity with 
universal covering lift f : A ^ A which has a non-trivial translation interval and 
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suppose a C A is an embedded arc joining the two boundary components of A. Then 
there exists m > such that for any 7 C A which is an embedded arc, disjoint from a, 
joining the two boundary components of A, every lift of f^{'j), \k\ > m must intersect 
more than one lift of a. 

Proof. Let a be a lift of a and let T be a generator for the group of covering transla- 
tions of A. We claim that there is an m > such that for all k > m, f^{6i) intersects 
at least 6 adjacent lifts of a. To show the claim consider the fundamental domain 
Dq bounded by a and T{a). If there are arbitrarily large k for which f^{o) intersects 
fewer than six translates of a, then for such A;, f'^^Do) would lie in an adjacent strip 
of five adjacent translates of Dq and the translation interval for / would have length 
0. This contradiction verifies the claim. 

There is a lift 7 of 7 that is contained in Dq. Let X be the region bounded by 
T~^(7) and T{'~f) and note that a G X. If the lemma fails then we can choose k > m 
and a lift h of f'' such that /i(7) is disjoint from T*(a) for all i ^ 0. It follows that 
h{X) is contained in U|^_2T-'(-Do) in contradiction to the fact that h{'j) intersects at 
least six translates of a. □ 

Lemma 7.3. Suppose f,g & Z{M) and there are trimmed level sets Ci e and 
C2 G C{g) such that there exist points Xi e M\ (Ci UC2), 1 < ^ < 4 with the following 
properties: 

(1) Xi,X2 lie in the same component of the complement of Ci and X3,X4 lie in the 
other component of this complement. 

(2) xijXa lie in the same component of the complement of C2 and X2,X4 lie in the 
other component of this complement. 

Then for some n > 0, the difjeomorphisms /" and gi" generate a non-ahelian free 
group. 

Proof. Let pi be a path in M \ C2 joining xi and X3; so /3i crosses Ci and is disjoint 
from C2. Likewise let P2 be a path in M \ Ci joining Xi and X2\ so P2 crosses C2 and 
is disjoint from Ci. (See Figure 1.) The level set C\ is an intersection 

00 

Ci = f|cl(5„), 

n=l 

where each is an /-invariant open annulus with cl(i?„_)_i) C and the rotation 
interval Bn) of the annular compactification f^ of is non-trivial (see section 
15 and the proof of Theorem 1.4 in [6]). For n sufficiently large cl(S„) is disjoint 
from (52 and {xj}^^^. Let Ai be a choice of B^ with this property. 

We may choose a closed subarc «i of /3i whose interior lies in Ai and whose end- 
points are in different components of the complement of Ai. We will use intersection 
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Figure 1: The curves and ^2 



number with cti with a curve in Ai to get a lower bound on the number of times that 
curve "goes around" the annulus Ai. 




Figure 2: The curves ai, 0^2, 70 and 71 
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Similarly the level set C2 is an intersection 

00 

n=l 

where each B'^ is a (7-invariant open annulus with cl(i?^_^^) C B'^ and the rotation 
interval p{g. fi'J of the annular compactification g^. of is non-trivial. We construct 
A2 and the arc ^2 in a fashion analogous to the construction of Ai and ai. By 
construction cti is disjoint from A2 and crosses Ai while 0:2 is disjoint from Ai and 
crosses A2. (See Figure 2.) 

Note that any essential closed curve in Ai must intersect ai and must contain 
points of both components of the complement of C2. To see this latter fact we note 
that we constructed ai to lie in one component of the complement of C2 but we could 
as well have constructed a'l in the other component of this complement. Any essential 
curve in Ai must intersect both ai and a[. Similarly any essential curve in A2 must 
intersect 0:2 and must contain points of both components of the complement of Ci. 

There is a key consequence of these facts which we now explore. Let 70 be an arc 
with interior in Ai, disjoint from A2 U ai and with endpoints in different components 
of the complement of A^. Replace / and g by /™ and g"^ where m and m' are the 
numbers guaranteed by Lemma 7.2 for / and g respectively. Then we know that for 
k ^ the curve f'^i'jo) must intersect more than one lift of the arc ai in the universal 
covering Ai. 

It follows that /'^(7o) contains a subarc whose union with a subarc of ai is essential 
in Ai. Hence we conclude that f''{jo) contains a subarc crossing A2, i.e. a subarc 71 
whose interior lies in AinA2 (and hence is disjoint from a2), and whose endpoints are 
in different components of the complement of A2. (See Figure 2.) Since we replaced g 
by g"^ above we know that for A; 7^ the curve g'^i'ji) must intersect more than one 
lift of the arc a2 in the universal covering ^2. 

We can now construct 72 in a similar manner but switching the roles of / and 

g, cti and 0:2, and Ai and A2. More precisely, for any k the curve g''{'ji) contains 
a subarc whose union with a subarc of a2 is essential in A2. It follows that g'^i'fi) 
contains a subarc 72 whose interior lies in AinA2 and whose endpoints arc in different 
components of the complement of Ai. Note that 72 is a subarc of 5'™/^(7o)- 

We can repeat this construction indefinitely, each time switching / and g. Hence 
if we are given h — g'^^ f^^ . . . g^'^f'^'' and rrii ^ 0, ki ^ we can obtain a non-trivial 
arc 72n which is a subarc of /i(7o)- Since 72n G Ai n A2 and 70 fl ^2 = it is not 
possible that h — id. But every element of the group generated by / and g is either 
conjugate to a power of /, a power of g, or an element expressible in the form of 

h. Hence we conclude that the group generated by / and gi is a non-abelian free 
group. □ 

Proof of Theorem 1.12. Suppose / G G fl Z{M) and U G A{f) is a maximal 
annulus for /. One possibility is that there is a finite index subgroup Gq of G which 
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preserves infinitely many of the trimmed rotational level sets for / which lie in U . In 
this case Lemma 7.1 implies Gq is abelian and we are done. 

If this possibility does not occur, we claim that there exists a trimmed level set 
C inU and ho e G such that ho{C)nC ^ C but /io(C) n C 7^ 0. If this is not the 
case then for every h e G and every C either h{C) = C or h{C) flC — It follows 
that the G-orbit of C consists of pairwise disjoint copies of C. Since elements of G 
preserve area this orbit must be finite and it follows that the subgroup of G which 
stabilizes C has finite index. If we now choose Cq and Cq, two trimmed level sets in U 
and let Go be the finite index subgroup of G which stabilizes both of them, then the 
annulus Uq which they bound is Go- invariant. Now if G is any trimmed level set in Uq 
then its Go-orbit lies in U and we conclude from area preservation that g{C) n G 7^ 
for all g e Gq. If for some g and G we have g{C) nC ^ C we have demonstrated 
the claim and otherwise we are in the previous case since Go preserves the infinitely 
many trimmed level sets lying in Uq. 

So we may assume that the claim holds, i.e., that there is /io £ G and a trimmed 
level set Gi for f m U such that /lo(Gi) n Gi 7^ Gi and /lo(Gi) n Ci ^ $. Let 
G2 = ^o(Gi) and g — hofh^^. Since Gi 7^ G2 and each is the common frontier of its 
complementary components, it follows that each of the complementary components 
of Gi has non-empty intersection with each of the complementary components of G2. 
Hence we may choose points Xi E M \ (Gi U G2), 1 < ^ < 4 satisfying the hypothesis 
of Lemma 7.3 which completes the proof. □ 
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